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(57) Abstract: A television having a processor that increases the 
viewability of a picture-in-picture display (PIP) by producing a track- 
ing window in a primary image display area and tracking a corre- 
sponding image portion in the PIP. The television has a user input 
device that receives a user input to enable a user to move the track- 
ing window to any given portion of the primary image display area. 
The user input also enables a user to adjust the size of the tracking 
window and to adjust a zoom level of the PIP. 
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This invention generally relates to a method and device to enhance home 
television usage. Specifically, tiie present invention relates to a picture-in-picture display. 

5 It is very common for televisions to have a capability of displaying more than 

one video display on the television display at the same time. Typically, the display is 
separated into two or more portions wherein a main portion of the display is dedicated to a 
selected video data stream (e.g., a given television channel). A second video data stream is 
simultaneously shown in a display box that is show as an inset over the display of the first ^ 

10 data stream. This inset box is typically denoted as a picture-in-picture display ("PIP"). This 
PIP provides the functionality for a television viewer to monitor two or more video data 
streams at the same time. This may be desirable for instance at a time when a commercial 
segment has started on a given television channel and a viewer Avishes to "surf* additional 
selected television channels during the commercial segment, yet does not vdsh to miss the 

1 5 retum fi:om the commercial segment. At other times, a viewer may wish to search for other 
video content or just view the other content without missing content on another selected 
channel. 

In any event, PIP has a problem in tiiat the PIP is typically shown in an inset 
box that is substantially smaller than the entire television display. Techniques are known to 

20 produce the reduced size PIP, however, with this reduction in size tiiere is a reduction in the 
viewability of the displayed image. People buy larger television sets to increase the size of 
displayed images so that the images are in effect, more viewable. Yet, vsdth the addition of 
PIP features into televisions, a smaller, less viewable image has been introduced on to these 
larger displays. These reduced PIP images produce a marked reduction in the ability of a 

25 viewer to follow events or content flow of selected content. 

Accordingly, it is an object of the present invention to overcome the 
disadvantages of the prior art. It is a fiirther object of the present invention to provide a 
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method and device for increasing the viewability of PIP images. To this end, the invention 
provides what is defined by the independent claims. The dependent claims define 
advantageous embodiments. 

5 

In a preferred embodiment, a television has a processor that increases the 
viewability of a picture-in-picture display (PEP) by producing a tracking window in a primary 
image display area and thereby, track a corresponding image portion in tiie PIP. The 
television has a user input device that receives a user input to move the tracking window to 

10 any given portion of the primary image display area. The user input also enables a user to 
adjust the size of the tracking window and to adjust a zoom level of the PIP. 

In one embodiment, the processor captures an image frame in the primary 
image display area for placement of the tracking window. The processor may utilize known 
object recognition techniques to identify the image portion corresponding to the position of 

1 5 the>tracking window. These techniques may be fiirther utilized to size the tracking window 
to the identified image portion. 

The following are descriptions of embodiments of the present invention that 
20 when taken in conjxmction with the following drawings will demonstrate the above noted 
features and advantages, as well as fijrther ones. It should be expressly imderstood that the 
drawings are included for illustrative piirposes and do not represent the scope of the present 
invention. The invention is best understood in conjunction with tihie accompanjdng drawings 
in which: 

25 FIG. 1 shows an illustrative image tmcking system in accordance wilh an 

embodiment of the present invention; 

FIG, 2A shows an enlarged view of a display at an initial time Tl containing 
illustrative objects than can be tracked in accordance with an embodiment of the present 
invention; and 

30 FIG. 2B shows an enlarged view of the display at a time T2 subsequent to the 

initial time Tl. 
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In the discussion to follow, certain terms will be illustratively discussed in 
regard to specific embodiments or systems to facilitate the discussion. However, as would be 
readily apparent to a person of ordinary in the art, these terms should be imderstood to 
encompass other similar applications and embodiments wherein the present invention could 
5 be readily applied. 

FIG. 1 shows an illustrative image tracking system 100 in accordance with an 
embodiment of the present invention including a display 110, operatively coupled to a 
processor 120, and a remote control device 130. The processor 120 and the remote control 
device 130 are operatively coupled as is known in the art via an infrared (IR) receiver 125, 

1 0 operatively coupled to the processor 1 20, and an IR transmitter 131, operatively coupled to 
the remote control device 130. 

The display 110 may be a television receiver or other device enabled to 
reproduce audiovisual content for a user to view and listen to. The processor 120 is operable 
to produce a picture-in-picture display (PIP) on the display 1 10 as is know by a person of 

15 ordinary skill in the art. Further, the processor 120 is operable to provide a PIP display in 
accordance with an embodiment of the present invention. 

The operation of the illustrative system shown in FIG. 1 will be described 
herein below with reference to FIGs. 2A, 2B. The remote control device 130 contains 
buttons for operation in accordance with the present invention. Specifically, the remote 

20 control device 130 contains a PIP button 134, a swap button 132, a tracking window button 
133, tracking window sizing buttons 136A, 136B, tracking window zoom buttons 135 A, 
135B, and tracking window position control buttons 137A, 137B, 137C, 137D. The PIP 
button 134 initiates a PIP function to open a PIP 21 OA (e.g., see, FIG. 2A) on the display 110 
and the swap button 132 swaps each of the images shown on the PIP 210A portion and on a 

25 primary display portion 2 1 OB of the display 1 1 0 as is known. 

FIG. 2A shows an enlarged view of the display 1 10 at an initial time Tl 
containing illustrative objects than can be tracked in accordance with an embodiment of the 
present invention. Specifically, objects such as an airplane object 220, a truck object 230, a 
walking person object 240, and a running person object 250 are shown in the primary display 

30 portion 210B. It should be expressly vinderstood that although the present invention is 

described with regard to objects, a person of ordinary skill in the art would readily appreciate 
that other picture portions may be tracked and still be well within the boimds of the present 
invention. 
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Specifically, it is known in the art that color(s), contrast, object detection, 
motion, etc. may be utilized by a processor, such as the processor 120, for tracking a given 
portion of a video image amongst a sequence of frames. Further, it is known that even 
multiple color portions may be tracked simultaneously amongst a sequence of frames. Much 
5 attention in the prior art has also been paid to the tracking of a himian body amongst a 

sequence of frames. Any of the above-mentioned systems for tracking a given portion of a 
video image from a video data stream (e.g., a television data stream) may be utilized in 
accordance with the present invention. 

The operation of a tracking system in accordance with an embodiment of the 
10 present invention can be described as follows. First, a user initiates an action, such as 

depression of the tracking window button 133, to bring up a tracking window 260, In FIG. 
2A, the tracking window 260 is shown centered within the primary display portion 21 OB. 
The centered position of the tracking window 260 may be the initial position after the user 
depresses tracking window button 133 or it may be the position after the user has positioned 
: 15 the tracking window 260 utilizing the window position control buttons 137A, 137B, 137C, 
137D, The window position control buttons 137A, 137B may respectively adjust the 
tracking window 260 up and down. The window position control buttons 137C, 137D may 
respectively adjust the tracking window 260 left and right. 

Next, the user sizes the tracking window 260 utilizing the window sizing 
20 buttons 136A, 136B. For example, the window sizmg button 136A may enlarge the size of 
the tracking window 260 and the window sizing button 136B may reduce the size of the 
tracking window 260. By sizing the tracking window 260, the user adjusts the portion of the 
video image shown that the system, in accordance with the present invention, will track in the 
PIP 210A. 

25 It should be noted that although the tracking window is illustratively shown 

and described as a window surrounding an object that is to be tracked, in fact the tracking 
window may not surround the object. In some embodiments, the tracking window may 
actually be a single indicated point, or a grouped plurality of points on the primary display 
area 21 OB. In these embodiments, the tracking window may not surround an object or region 

30 of interest. In either case, the device in accordance with the present invention may utilize 
well known object detection and identification techniques to determine the bounds of a 
selected object. In this way, the device in accordance with the present invention may 
determine the bounds of a selected object, region, etc. for tracking in the PIP 21 OA. 
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The single indicated point may be a result of reducing the size of the tracking 
vsdndow 260 utilizing window sizmg button 136B, or it may be a result of a predetermined 
initial condition for the tracking window 260. Accordingly, the tenn tracking window as 
utilized herein should be xmderstood to encompass these and other related visual indications 
for facilitating a user to select what they wish to track. Clearly, the tracking window 260 is 
envisioned to encompass any indication that may range from a single point to any portion of 
the primary display area 21 OB and may take on any suitable shape. For instance, in one 
embodiment, after a user selects a pomt for the tracking window 260, the system in 
accordance with the present invention may perform an object recognition step and 
automatically size the tracking window to the area of interest. In this case, the tracking 
window may be presented visually as si highlighting or other indication around the 
recognized object. Automatically placing the visual indication of the tracking window 260 
aroimd a selected portion of the video image may facilitate the user identifying and 
repositioning the tracking window in the event that an incorrect portion of the video image is 
recognized by the system. 

Next, the system tracks the selected portion of the video image in the PIP 
21 OA. The truck object 230 is shown coincidentally centered within the tracking window 
260 in the primary display portion 210B. However, as should be clear from the discussion 
above, the tracking window 260 may be positioned on any portion of the primary display area 
21 OB. In accordance with the present invention, the tmck object 230 is tracked and is shown 
centered in the PIP 210A. Further, the PIP 210A illustratively shows a portion of the airplane 
object 220 merely due to the airplane objects relative position with regard to the tracked truck 
object 230. 

As should be clear from FIG. 2A, the PIP 210A is not an ordinary display of 
the television image as is known in the art. In accordance with flie present invention, the PIP 
21 OA shows an enlarged portion of the television image with a selected object (e.g., the track 
object 230) tracked and centered within the PIP 21 OA. As discussed above and as should be 
readily appreciated, other portions of the television image may also be tracked m accordance 
with a present invention by the user selecting the other portions of the television image 
utilizing the tracking window 260. 

Next, the user may optionally adjust the degree of zoom that is shown in the 
PIP 210 through use of the zoom buttons 135A, 135B. Electronic zoom techniques are 
known in the art wherein additional pixels may be interpolated to produce a zoomed image 
(e.g., an enlarged image). The zoom button 135A may increase the degree of zoom of the 
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selected portion of the television image shown in the PIP 21 OA, thereby increasing the size of 
the selected portion in the PIP 21 OA. In accordance with the present invention, this increases 
the viewability of a portion of the television image of interest. In one embodiment, 
depression of the zoom button 135 A may also increase the size of the PIP 21 OA. The zoom 
5 button 135B may decrease the degree of zoom of the selected portion of the television image 
shown in the PIP 21 OA and/or decrease the size of the PEP 21 OA. 

FIG. 2B shows an enlarged view of the display 1 10 at a time T2 subsequent to 
the initial time Tl shown in FIG. 2 A. In FIG. 2B, at the time T2, the objects shown have 
changed position with regard to the objects position shown at the time Tl . However, in 

1 0 accordance with a present invention, the truck object 230 is still shown centered within the 
tracking window 260 and the PIP 21 OA even though the relative position of the truck object 
230 has moved to the right of the primary display portion 21 OB. Further, the position of the 
airplane object 220 has changed relative positions in the primary display 210A as well as in 
the PEP 210B since the airplane object 220 has not been selected by the user for tracking. . 

1 5 Next, the system in accordance with the present invention will continue to 

track the selected portion of the television image as long as the selected portion is within the 
primary display portion 21 OA. Tracking of the selected portion is discontinued due to the 
selected portion no longer being present in the primary display portion 21 OA. Of course, 
tracking of the selected portion may be discontinued at anytime by the user once again 

20 depressing the tracking window button 133. 

In accordance with another embodiment of the present invention, when the 
user &st depresses the tracking window button 133, a frame of the displayed television 
image is captured by the processor 120 and is continuously displayed on the display 110. In 
this way, placement of the tracking window. 260 is facilitated in the event of a quickly 

25 changing television image, such as a television image of a football game, etc. Once the user 
selects the portion of the video image that the user wishes to track, the primary display area 
may revert back to displaying the video data stream. 

Finally, the above-discussion is intended to be merely illustrative of the 
present invention. Numerous altemative embodiments may be devised by those having 

30 ordinary skill in the art v^thout departing from the scope of the following claims. In the 

claims, any reference signs placed between parentheses shall not be construed as limiting the 
claim. The word "comprising" does not exclude the presence of elements or steps other than 
those listed in a claim. The word "a" or "an" preceding an element does not exclude the 
presence of a plurality of such elements. The invention can be implemented by means of 
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hardware comprising several distinct elements, and by means of a suitably programmed 
computer. In the device claim enumerating several means, several of these means can be 
embodied by one and the same item of hardware. The mere fact that certain measures are 
recited in mutually different dependent claims does not indicate that a combination of these 
measures cannot be used to advantage. 
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1 . A television comprising: 

a display (110) having a primary image display area (21 OB) and a picture-in-picture 
display area (PIP) (210A); and 

a processor (120) operatively coupled to the display (1 10) and configured to produce 
5 a tracking window (260) in the primary image display area (2 1 OB) and to track a 

corresponding image portion in the PIP (21 OA) in response to user input. 

2. The television of Claim 1 , comprising a user input device (130) configured to 
receive a user input and to provide signals to the processor (120) to move the tracking 

1 0 window (260) to any given portion of the primary image display area (2 1 OB). 

3. The television of Claim 1 , comprising a user input device (1 30) configured to 
receive a user input and to provide signals to the processor (120) to adjust the size of the 
tracking window (260). 

15 

4. The television of Claim 1, comprising a user input device (130) configured to 
receive a user input and to provide signals to the processor (120) to adjust a zoom level of the 
PIP (21 OA). 

20 5. The television of Claim 1, wherein the processor (120) is configured to capture 

an image frame in the primary image display area (21 OB) for placement of the tracking 
window (260). 

6. The television of Claim 1 , comprising a user input device (1 30) configured to 

25 receive a user input and ito provide signals to the processor (120) to adjust the position of the 
tracking window (260) in the primary display area (21 OB), wherein the processor (120) is 
configured to identify the image portion corresponding to the position of the tracking window 
(260). 
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7. The television of Claim 6, wherein the processor (120) is configured to size 
the tracking window (260) to the identified image portion. 

8. An image signal processing comprising: 

5 - placing a tracking window (260) over a portion of a video image displayed in a 
primary display area (21 OB) of a television display; 

tracking a corresponding portion of the video image in a picture in picture display 
(PIP)(210A). 

10 9. The method of Claim 8, comprising capturing an image frame in the primary 

image display area (21 OB) for placement of the tracking window (260). 

10. A software program sequence stored on a computer readable medium, the 
soflware program sequence comprising program portions for enabling a processor to carry 

1 5 out the method of claim 8 . 

11. A user input device (1 30) configured to receive a user input and 

to provide signals to move a tracking window (260) to any given portion of a primary 
image display area (21 OB); and/or 
20 - to provide signals to adjust a size of the tracking window (260), and/or 
to provide signals to adjust a zoom level of a PIP (2 1 OA). 
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